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ABCmouse English Is Highly Effective in 
Helping Young Chinese Children Learn 
English: A Randomized Control Trial 


Key Findings 


e By using ABCmouse English for about 30 hours over 
six months, beginner English learners improved their 
English language skills nearly threefold. 


e Treatment group children made significantly 
greater gains in English language knowledge 
compared to the control group. 


e Treatment group children’s level of engagement 
with ABCmouse English was statistically 
significantly higher than that of their control 
group peers with their assigned apps. 


e Treatment group parents reported improvements 
not only in their children’s English language 
skills, but also in their confidence, interest, and 
motivation to learn English. 


Overview 


In many parts of the world where English is not a 
primary language, parents consider English language 
skills critical to their children’s success in life. This is 
particularly true among parents in China, who exhibit 
a very significant demand for systems, teaching 
approaches, and products that are designed to help 
children learn English.' Although compulsory English 
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education in China does not begin until the third year 
of primary school, an increasing number of schools are 
introducing it earlier in their curriculum, and parents are 
following suit.? 


In recent years, the number and variety of digital 
education programs designed to facilitate learning in 
various subject areas has grown rapidly. Language 
learning is especially well-suited for digital game-based 
learning, given the immersive environment, lower 
anxiety, and increased use of the target language 
afforded in interactive games.’ One digital learning 
program available to young Chinese children seeking 
to develop their English language skills is Age of 
Learning’s ABCmouse English, which teaches English 
as a foreign language (EFL) to children (ages 3-8) in 

a developmentally appropriate, immersive English 
language environment, in the same natural sequence as 
children learning their native language. 


For this study, Age of Learning partnered with OneLeap, 
a design and research consultancy in China. Age of 
Learning researchers designed the study and worked 
with OneLeap to recruit, communicate with the 
participants, and arrange the logistics associated with 
data collection. Age of Learning also conducted the 
data analyses and interpretations of results. A primary 
goal of the study was to understand the extent to which 
ABCmouse English, a self-guided digital learning app, 
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can help 5- and 6-year-old Chinese children develop 
English language skills. 





ABCmouse English is a self-guided digital learning 
program that incorporates empirically supported, 
effective practices from second language instruction. 

A team of language acquisition researchers, teachers, 
curriculum specialists, and developers used the 
Common European Framework of Reference (CEFR) 
standards, the American Council on the Teaching of 
Foreign Language (ACTFL) guidelines, and second 
language research to create a program that presents 
content in the developmental sequence that language 
is acquired. The program is task- and activity-based, 
and it provides engaging opportunities for learners to 
interact with activities in authentic, meaningful contexts 
that include objects, events, and situations that children 
encounter in their everyday lives (e.g., animals, families, 
foods).* Learners receive scaffolded target language 
input,° construct knowledge through trial and error, and 
receive timely and developmentally appropriate implicit 
and explicit corrective feedback that is embedded in 
activities. Additionally, the self-guided Learning Path 

in the program leads learners through progressively 
more challenging material and is designed to encourage 


children to become active, independent learners. Finally, 





Figure 1. Bubble Popper Game: Player hears the word fish. 
The task is to tap on all of the bubbles that contain fish and 
ignore the bubbles that contain distractors. 


given the great deal of research linking motivation and 
second language (L2) learning outcomes, the program 











Figure 2. Talk Time Game: a) Player hears a conversation 
between two children. b) Player selects a correct answer 
among three sentences produced by three children. c) Player 
is asked to produce an answer to the question (microphone). 
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was designed first and foremost to promote learner 
engagement and motivation.’ 


The participants in this study used the first two levels 
of the program, which targeted a total of 165 words 
and 15 sentences. The themes covered in these levels 
include animals, common objects, food, places, colors, 
the numbers 1-15, shapes, people, feelings, body parts, 
clothing, pronouns, and classrooms. Figures 1-2 show 
screenshots of sample games that students played in 
the program. 


Participants 


A total of 122 children and their parents participated 

in the study between December 2018 and June 2019. 
Participants resided in Hangzhou, the fourth largest 
metropolitan area in China, and we used a list of 
screening questions to identify children between 5 and 
6 years old at the start of the study, with little to no prior 
English knowledge (i.e., knowing fewer than 20 words 
in English). Children whose parents were professionals 
in industries with a substantial amount of English usage 
(e.g., English teaching, marketing) were not eligible 

for participation. There were no significant differences 
between groups in terms of parents’ educational levels 
(62.1% of control and 60.7% of treatment group parents 
reported having completed junior college) or income 
(51.7% of control versus 50.8% of treatment group 
parents reported monthly income of USD 1,423.00- 
2,846.00). The final analytic sample included 113 children 
(63 treatment, 50 control), excluding 6 (1 treatment, 

5 control) who enrolled in an English class during 

the study; 2 treatment children who were unable to 
attend an assessment; and 1 control child who was not 
interested in using the assigned app. 


Design and Procedures 


We used two measures of English language proficiency, 
one external and one internal, to assess children’s 
language skills at the start and at the end of the six- 
month period. After balancing pretest scores, age, 

and gender, we randomly assigned 66 children to the 
treatment group and 56 children to the control group. 
We asked the treatment group children to use the 


ABCmouse English program for a minimum of 15—20 
minutes daily for six days a week for 26 weeks. To 
ensure all participants met the minimum threshold for 
usage each week, we monitored the usage data on a 
weekly basis via surveys completed by parents and 

by reviewing the back-end game data. When children 
did not meet the usage goals, we reached out to their 
parents through our research consultants in China and 
offered incentives to those who made the greatest 
improvements in terms of making up for missed time. 
For the control group children, we assigned digital math 
learning apps (Xiaobanlong, Youdao Math, and Miao 
Kids Math, none of which have content in English) and 
asked them to use the app for the same amount of time 
over the six months. Additionally, all parents completed 
an end-of-study survey, and 14 treatment parents 
participated in individual interviews or focus groups. 


The external measure of children’s English skills was 
the IDEA Language Proficiency Test (IPT),° a long- 
standing, widely-used assessment of overall English 
language skills that has been found to be accurate 
and reliable in studies involving speakers of Spanish, 
Chinese, Japanese, and Korean.? We used the Pre- 
IPT Oral Proficiency Test, which is designed for young 
learners between ages 3 and 5, and it involved an 
examiner asking 10 questions (targeting vocabulary, 
grammar, comprehension, and language functions) in 
English using a storyboard and cardboard pieces. The 
internal test was one that Age of Learning curriculum 
specialists and assessment experts created, and it 
targeted linguistic forms that children would have been 
naive to at the beginning of the study but learned 
through the game over the course of the study. This 
assessment included a total of 25 questions divided into 
five subsections (Vocabulary Identification, Listening 
for Meaning, Speech Production, Conversation, and 
Pronunciation). Two bilingual assessors, blind to the 
treatment or control condition assignments of children, 
administered the assessments in English. 


On average, treatment children used ABCmouse 

English for 29.29 hours (SD = 8.21) across 118.75 days 
(SD = 27.20) and completed 3,015 questions (SD = 1,096). 
Control students used their apps, on average, for 11.99 
hours (SD = 2.32) during the same period. 
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Results 


Finding 1. By using ABCmouse English for about 30 hours, beginner English learners improved their English 
language skills nearly threefold. 


We created an overall score of English language skills by creating a weighted average of the children’s scores on 
the external and internal assessments. As shown in Figure 4, by posttest, treatment group children significantly 


outperformed their control group peers on the overall measure of English language skills (¢(111) = 11.61, p < .001, 
Cohen’s d = 2.17). 
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Figure 4. Weighted average of pre- and posttest scores on the IDEA Language Proficiency Test and internal assessment of language 
skills. Lighter shades represent pretest scores; darker shades represent posttest scores. 


Finding 2. Students using ABCmouse English made significantly greater gains in English language knowledge 
compared to the control group. 


The pre- and posttest comparisons of individual subsections of the internal assessment showed that treatment group 
children made statistically significant improvements on each of the language skills assessed (Figure 5). The difference 


between the treatment group’s and the control group’s posttest scores on each section was Statistically significant at 
p < .001. Effect sizes (Cohen’s d) ranged from 0.68 to 2.18. 
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Figure 5. Pre- and posttest scores on subsections of the internal assessment. Vocabulary Identification (¢(111) = 11.62, p < .001, Cohen’s 


d = 2.18); Listening for Meaning (t(111) = 7.62, p < .001, Cohen’s d = 1.43); Speech Production (t(111) = 9.59, p < .001, Cohen’s d = 1.80); 
Conversation (t(111) = 3.62, p < .001, Cohen’s d = 0.68); Pronunciation (¢(111) = 6.41, p < .001, Cohen’s d = 1.21). 
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Finding 3. Across nearly all 26 weeks of the study, treatment children’s level of engagement with ABCmouse 
English was statistically significantly higher than that of their control group peers with their assigned apps. 


On weekly surveys, parents reported whether their children were “never,” “rarely,” “occasionally,” “very frequently,” or 
“always” engaged while using their learning apps (Figure 6). Except for weeks 1 and 9 of the study (Lunar New Year), 
the majority of treatment parents consistently reported that their children were “very frequently” or “always” engaged 
when using ABCmouse English. The difference between the two groups’ engagement levels is statistically significant 
(p < .05—p < .001). 
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Figure 6. Parents’ average weekly report of their children’s engagement level by group. 


In interviews, parents further elaborated on how ABCmouse English engaged their children in learning English. 


“We used to teach him some simple words, but he didn’t show a strong interest... but he fell in 
love with the lessons on ABCmouse. Now he has learned to express himself in English when he 
sees something he likes. | see his passion now.” 

—Mother of a 6-year-old boy 


“Over the past six months, my daughter has enjoyed English learning and the app. She has 
not used it these [last] few days, and she asked me when she could study with the app. It has 
become a must in her life.” 
—Mother of a 6-year-old girl 


End-of-study parent surveys also revealed that parents viewed ABCmouse English as an effective resource to help 
children get an early start in English language learning. The native English speakers featured in the activities offered 
Chinese children opportunities to listen to and learn from the accurate pronunciations of English words. The design of 
the game, emphasizing authentic communication, was particularly valuable in the EFL context where limited input in 
the target language is available in learners’ everyday lives. 


Finding 4. Treatment parents reported that their children not only improved in their English language skills, but 
also that they became more confident, interested, and motivated to learn English. 


Feedback collected from parents through end-of-study surveys corroborated the students’ assessment results. As 
shown in Figure 7, most treatment group parents indicated that they observed improvements in their children’s ability 
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to speak, read, understand, and pronounce English words and sentences. Similarly, the majority of treatment parents 
reported that their children became more confident, interested, and motivated to learn English and more self-directed 
and persistent as learners (Figure 8). 


My child has made improvements in... 


speaking words or sentences in English 87 


reading or identifying words or sentences in English 81 
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pronouncing words in English 75 
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Figure 7. Percent of treatment group parents reporting “strongly agree” or “agree” with each statement. 
| believe that ABCmouse English has helped my child... 
become more confident in English 79 
become more interested in learning English 76 
become more motivated to learn English 71 
become more self-directed as a learner 68 
become more persistent as a learner 64 
(0) 20 40 60 80 100 
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Figure 8. Percent of treatment group parents reporting “strongly agree” or “agree” with each statement. 





resource. This was especially the case for those who 


Conclusion 


may not have the means or the time to take their 


This is the first efficacy study of an ABCmouse English 
product designed specifically for young learners of 
English in China. By using the self-guided learning 
program consistently over six months, beginner English 
learners made significant gains in their English language 
skills. Parents greatly valued the opportunity to have 
their children engage in activities and interactions with 
the native-English-speaking instructors in ABCmouse 
English. They indicated that it is a cost-effective learning 


children to English language classes or provide other 
input for them, but who nevertheless want their children 
to have an early start in English. The fact that ABCmouse 
English also helped children stay engaged in learning, 
while developing an interest in and a motivation 

for learning English, also makes it valuable, since 
engagement, interest, and motivation are important 
prerequisites for learning. 
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